Mobile phones with digital cameras have become a pervasive type of mobile devices. Worldwide smartphone sales have surpassed one billion units for the first time in 2014. According to market forecasts, in 2014-2018, annual smartphone shipments are expected to grow from 1.28 to 1.87 billion units. Over 80% of all mobile phones will be arriving to customers with embedded digital cameras. Market research also indicates that photographing with phone cameras ranks first among the most popular activities of smartphone owners. According to recent polls, photographing with phone cameras was employed by 82% of users, exceeding even the previously most popular activity, texting, which was utilized by 80% of the owners. Other studies indicate that every year over a quarter of all photographs are taken with smartphones; the total count of images posted on social photo-sharing sites and taken with smartphones has exceeded the cumulative count of posted photographs taken with all other types and models of non-smartphone equipment.
Hundreds of millions smartphone users are daily employing smartphone cameras in their business offices and homes and are combining their digital habits with the conventional lifestyle where paper media retains its significant role in the everyday information flow of businesses and households. Pages from books and magazines, printed newspaper articles, receipts, invoices, checks, tax and other forms, printed reports, business cards, handwritten notes and memos on legal pads, in specialized Moleskine notebooks, on sticky notes or easels, and many other types of printed and handwritten documents are increasingly benefiting from digital capturing. Thus, a recent survey of smartphone usage by millennials has revealed that 68% of survey respondents have been introduced to mobile information capturing via mobile check deposits, while 83% share an opinion that mobile capture will be part of all mobile transactions within the next few years.
The role of digitizing and capturing of paper based information has increased with the arrival of unified multi-platform content management systems, such as the Evernote service and software developed by Evernote Corporation of Redwood City, Calif. Growing camera resolution, quality of photographs and feature sets of smartphone cameras allow users to capture scenes with rich content and multiple objects.
In recent years, many software applications for advanced real-time processing of photographs made with smartphone cameras have been developed. These applications may improve various aspects of image quality, fix perspective distortions, identify different types of objects in photographed scenes, etc. Combined with cloud services or working standalone, software applications may further recognize objects such as barcodes, text and faces and assign alphanumeric values or names to objects. Progress has also been made in automatic scanning of paper documents using smartphone cameras. For example, Scannable software by Evernote Corporation may identify and automatically capture document pages in complex scenes with perspective and other types of image correction or retrieve content layout within document pages.
Notwithstanding significant progress in pre-processing, correction and recognition of scenes captured with smartphone cameras, interaction levels between the smartphone and the user in the photographing process remain insufficient. Users have limited capabilities to select a sub-scene or a set of dedicated objects from a complex scene, avoid capturing of unnecessary objects, etc. Additionally, when a fragment of a paper document is captured, users don't gain access to a full document, although on many occasions, an online copy of the full document exists.
Accordingly, it may be useful to develop efficient mechanisms for interacting with a user at the time of capturing photographs with smartphone cameras and for supplementing a captured document fragment with a full online copy thereof.